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This thesis offers, to those concerned with analysis of modern 
weapons systems, a general methodology for devising appropriate and 
meaningful measures of effectiveness. This methodology does not 
include a specific model for "plugging in" system parameters and 
mechanically "grinding out" system effectiveness. It is Intended 
only as a general "plan" through which the researcher can channel his 
own judgment and experience. The primary purpose of this plan is to 
guide the researcher through a logical transition from a purely 
subjective, and more or less vague, "concept of effectiveness to a 
useable and more explicit formulation. 

Effectiveness is modeled as that single system characteristic 
positioned at the apex of a characteristic "pyramid". This pyramid 
is constructed with "layers" of progressively fewer and more subjective 
characteristics. Mathematical properties of measurements appropriate 
to these characteristics are discussed as a function of the Intended 
use of the effectiveness measurement. The type of measurement required 
to meet an analysis objective is dependent on the objective function 
or optimization criteria chosen. Because of this fact, the types of 
measurements have been classified into four scales and each scale 
related to a particular type of objective function. 
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Introduction. 



This thesis is principally concerned with the formulation of 
an expression for the relative or absolute magnitude of a system’s 
effectiveness. This "expression of magnitude" is the essence of the 
concept of measurement; but before attempting any plan for obtaining 
such an expression, it has appeared logical to explore some of the 
relevant properties of measurement. The particular manner in which 
this expression of magnitude is accomplished depends on its ultimate 
use in the analysis. In consideration of this fact, it has further 
appeared logical to discuss and catalogue different types of analyses 
and objective functions and relate them to types of measurement. 

The main feature of this thesis is the "characteristic pyramid" 
described in section 7.1. The reader may wish to refer to this section 
initially so as to understand better the direction of the preceding 
sections, or to make a brief appraisal of the model’s applicability to 
his own area of interest. The organization of this thesis is directed 
to that reader interested in the general concept of system effective- 
ness. Those seeking a specific model for system effectiveness should 
appraise section 7.1 and utilize other sections as a reference to 
those features of interest. 

1.1 Justification. 

The validity, and therefore the usefulness, of operations 
research depends upon the skill with which projects are designed, 
and particularly upon the shrewdness with which criteria (such as 
payoff and objective functions) are selected. This criterion problem 
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is often relatively neglected in operations research literature, 
and has apparently usually been "solved" in practice by assuming the 
first plausible payoff function that springs to mind; or if several 
spring to mind, by trying all of them and compromising (or letting a 
decision-maker compromise) among the results of alternative computa- 
tions, This problem is much too important for casual treatment. 
Calculating quantitative solutions using the wrong criteria is equiva- 
lent to answering the wrong question. If the methods of operations 
research are applied to the wrong criteria, its quantitative methods 
may prove worse than useless to its clients. 

The terminology, "effectiveness" and "system effectiveness" , is 
used extensively by the operations researcher, and is, more often 
than not, included in the objective function. This terminology is 
also applied as a "common denominator" through which to compare alter- 
native systems. A poor choice of a quantitative expression for 
effectiveness is equivalent to selecting the wrong criteria. 

From the multitude of characteristics exhibited by a system, 
there must be selected a set of factors that completely (or at least 
adequately) define the system's effectiveness. Typical characteristics, 
widely used, involve such terms as exchange rates, operability factors, 
probability of kill, and so on. 

All of these parameters used may be further divided into cases 
which are designed to cover range, altitude, speed, and the like. In 
studying any particular system, a particular selection of a system 
of characteristics must be made. The particular selection will depend 
on the particular situation under study. Some characteristics may be 
considered to be of greater importance than others, so that these 
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measures may be “weighted” in various ways. It often requires con- 
siderable judgment to select a workable and representative system of 
characteristics to be used as a basic for decision. 

Effectiveness is often conceived as a subjective quantity that 
is not ordinarily amenable to direct, physical measurement. Because 
of this, certain sets of characteristics that can be objectively 
described are chosen to indicate the subjective concept. This is, 
in effect, an extrapolation from the subjective concept to the objec- 
tive characteristics. Unless this extrapolation is logically sound 
and thoroughly understood, it is equivalent to an unintentional, re- 
definition of the system's mission. 

The effectiveness of a system depends on the mission of that 
system. If the mission is narrowly defined, it may be possible to 
measure the system's ability to achieve that mission directly through 
analytical models or experimental testing; however, narrow definition 
of systems and their missions tends to lead to a sub-optimization of 
the system with respect to broader, joint missions. Conversely, 
broader definition reduces the sub-optimization problem among systems, 
but increases the difficulty of obtaining a valid, analytical expression 
for the effectiveness. This suggests that one of the basic considera- 
tions in system definition should be an “optimization” of this measur- 
ability-sub-optimization relationship. 

To be useful as an analytical tool, the concept of effectiveness 
must be defined in such a way so as to permit its characterization 
through a set of physical measurements. The choice and number of the 
elements in this set is important in its own right, but the functional 
relationship between these elements and the concept of effectiveness 
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is the most elusive element of the analysis. 

The purpose of this thesis is to focus on the concept of 
effectiveness, to determine the types of measure and methods of 
measurement that are appropriate to various objectives, and to model 
the relationship between the elements of a system and the system's 
effectiveness . 

1.2 Formulating the problem 

Following is a listing of tasks that must be performed, in the 
general order presented, in the evaluation of a system's effectiveness: 

(a) Mission definition. 

(b) System definition. 

(c) Specification of relevant system characteristics. 

(d) Choice of an objective function. 

(e) Construction of a model that is consistent with the 
objective function. 

(f) Data acquisition. 

(g) "Fitting" of the data to the model. 

2. Measurement. 

1 

Measurement is defined by Peter Caws as "the assignment of 
particular mathematical characteristics to conceptual entitles in 
svich a way as to permit (1) unambiguous mathematical description of 
every situation Involving the entity, and (2) the arrangement of all 
occurrences of it in a quasi-serial order." The term "quasi-serial 
order" is taken to mean an order that determines, for any two occurren- 
ces, either that they are equivalent with respect to the property in 

^C, W, Churchman and P. Ratoosh, Measurement, Definition and 
Theories (Boston: John Whiley & Sons, Inc., 1962) p. 5 
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question or that one is greater than the other. This definition of 
measurement is applied to conceptual entities, which implies that 
before we can hope to measure effectiveness we must have some con- 
ceptual notion of effectiveness. A definition is some statement which 
sets the entity in unambiguous relation with other entities in the 
same or different groups. 

From these two definitions we can see that it may be possible 
to define effectiveness without providing a measurement for it, but it 
is impossible to measure effectiveness without first defining the 
concept . 

The "quasi-serial ordering" requirement can be accomplished in 

2 

four (4) ways which are classified by S. S. Stevens into four (4) 
scales of measurement. These four scales are described, along with 
their properties, in Table 1. 

This classification narrows our definition of measurement some- 
what, to those relationships between conceptual entities for which 
some property of the real number system can serve as a useful model. 
This restriction is implied if we say that a measurement is the assign- 
ment of numerals to aspects of those entities according to a rule. It 
is the particular property of the real number system, which we choose 
to serve as a model, that determines the properties and applicability 
of the measurement. Some "conceptual entities" can be measured on 
one type of scale and not on another, but the objective of the measure- 
ment may be consistent with only certain types of scales. 

^Ibid, p. 25 
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Any Number (x) on a Scale Can be Replaced By Another (x*) Without Affecting the Measurement 



2.1 Classification of effectiveness measurement objectives. 

When an attempt is made to find an expression, or measure, of 
a system's effectiveness, some specific use is planned for this 
measure. The nature of this "use” or objective determines the type 
of scale required to meet the objective. 

2.1.1. Comparison of several alternative systems or configurations 
to determine the least cost for a given effectiveness, i.e.. 

Minimize Cost 
so that 

Effectiveness = Constant 

Since there is no requirement for a knowledge of the absolute or 
relative magnitude of the effectiveness, the most general scale that 
will satisfy the objective is the nominal . 

2.1.2 Comparison of a discrete set of alternative systems or configu- 
rations to determine the greatest effectiveness for a cost less than 

or equal to some given amount, or conversely, i.e.. 

Maximize Effectiveness Minimize Cost 

so that or so that 

Cost Constant Effectiveness ^ Constant 

There is no requirement for a knowledge of the absolute magnitude of 
the effectiveness, but we must be able to order a set of discrete 
alternatives. The most general scale that will satisfy this objective 
is the ordinal . 

2.1.3 Comparison of a "continum" of system configurations to 
determine the greatest effectiveness for a cost less than or equal 



11 



to some constant, or conversely, l.e.. 



Maximize Effectiveness Minimize Cost 

so that or so that 

Cost ^ Constant Effectiveness ^ Constant. 

This objective is essentially the same as 2.1.2, but we are required 
to maximize a continuous function as opposed to choosing a maximum 
from a set of discrete quantities. 

2.1.4 An objective that falls naturally into this class is the 
determination of the rate of Increase (or decrease) in effectiveness 
as a function of cost. There is no requirement for a knowledge of 
the absolute magnitude of the effectiveness, but we must be able to 
compare amounts of increase in effectiveness as a function of the 
costs of those increases. The most general scale that will satisfy 
this objective is the interval scale . 

2.1.5 Determination of the absolute effectiveness of a system, so 
as to make utility comparisons of cost-effectiveness combinations. 

Compare: X Effectiveness and Y Cost 

with X* Effectiveness and y' Cost, 

where X*> X and Y*>Y , and make the "best" choice. 

This objective requires a measurement that provides a clear concept 
of the absolute magnitude of the effectiveness, and can be provided 
only by the ratio scale . 

3. Cost. 

All of the objectives discussed and classified above concern 
some type of cost-effectiveness trade-off. The term "cost" is not 
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restricted to the narrow definition of "dollar cost", but includes 
the general idea of resource requirements. Almost all system analysis 
is performed using the basic concept of some sort of trade-off between 
resource and effectiveness. To accomplish this trade-off, these two 
system parameters (cost and effectiveness) are separated, sometimes 
rather artificially, and envisioned as separate entities. This 
distinction is not, however, theoretically necessary. We could define 
the mission so as to include resource requirements and define the 
single resultant entity to be the system effectiveness. This concept 
of a system’s effectiveness would then imply only a single, unambiguous 
criterion for choice, namely, maximize effectiveness. This is not, 
however, the approach taken in this thesis, and the usual distinction 
between cost and effectiveness will be retained. 

4. A hypothetical example of the scales of measurement appropriate 
to various objectives. 

System definition: The warhead to be utilized on an existing 

missile . 

Mission: To produce a specified "overpressure 

distribution" over a circular area centered 
on the detonation point. 

Suppose that, originally, the only information available con- 
cerning the blast effects of various alternatives is that some produce 
identical distributions and others produce different distributions. 

From this primitive information we can place the effectiveness of the 
warheads on a nominal scale . If we establish a criterion of choosing 
the warhead with the least cost subject to its having an effectiveness 
equal to that of some arbitrarily selected warhead, we can use this 
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scale for selection of the best warhead. 

Suppose that through testing and/or analytical methods we now 
learn that the valxies of this distribution are related to some measur- 
able parameter of the warhead through an "increasing, monotonic 
function". If this is the only Information available concerning this 
function, we can place the effectiveness of different designs on an 
ordinal scale . If we establish a criterion of choosing the warhead 
having the greatest effectiveness for a cost less than or equal to a 
set amount (or conversely), we can pick the best warhead. 

Through further testing or analytical investigation we may learn 
the functional relationship existing between the design parameter and 
the effectiveness to within a multiplicative factor and an additive 
constant. 

This would be expressed as: 

Effectiveness *■ U*f (design parameter) + b, 

where f is known, and U and b are unknown constants. 

This scale is the most widely used for effectiveness because it requires 
no knowledge of the absolute magnitude of the effectiveness and the 
choice of units (U) is completely arbitrary. Potential energy offers 
an example of a physical quantity measured on this scale. 

If we are given the potential energy of object A as 10 "units" 
we must be furnished additional information concerning the reference 
point and the nature of the units before the statement is useful in 
itself . If, in addition, we are given the potential energy of object 
B as 20 "units", and told that the reference points and units of the 
two measurements are eqvial, we can make meaningful comparisons between 
the potential energy of the two objects. 
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Returning to our warhead example, we can make a direct measure- 
ment of the tons of explosives contained in the warhead. Assuming 
that f is the cube -root function, we construct an expression for the 
effectiveness of the warhead in the form: 



Effectiveness 



U*f (tons of explosive) 



1/3 



+ b. 



If design A has effectiveness = U*f(x) + b, and cost Y dollars 

design B has effectiveness = U«f(x ) + b, and cost Z dollars 

design C has effectiveness = U.f(x**) + b, and cost W dollars, 

we can make a statement of the form: 

For (Z-Y) dollars we can increase the effectiveness by U* f (x* ) -U* f (x) 
above the effectiveness of design A, and. 

For (W-Y) dollars we can increase the effectiveness by 

(U‘f(x") - U.f(x)) 

above that of design A. 

This statement provides the rate of increase of effectiveness as a 
function of the cost of that increase, and we may be able to make a 
choice between designs A, B, and C on the basic of this information. 

Finally, we may be able to obtain the exact value of the over- 
pressure as some function of design parameters. Since the mission of 
the warhead was defined "to produce a specified overpressure", we 
can obtain the overpressure of a particular design, compare it to the 
specified overpressure, and make a statement of the form: The effective- 

ness of the design is X% (or warhead A is X7o effective). An extremely 
important point to bear in mind, in connection with this example and 
in an actual analysis, is that we are expressing the ability of the 
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warhead to achieve its expressed mission and nothing else. If the 
mission of the warhead were intended as the accomplishment of certain 
damage to the target, there is no guarantee that the ability to achieve 
X% of the overpressure specified will, in fact, produce X% of the 
damage which would have resulted from the specified overpressure. 

This is a measurement that can be placed on the ratio scale and requires 
that the units of the quantity measured be identical (or differ by a 
known multiplicative factor) with the units expressing the mission 
attainment. In addition, it requires a common reference point which, 
in our example, is assumed to be the following: 

Zero overpressure corresponds to zero effectiveness. 

Having a measurement of effectiveness on this scale permits a 
criterion of "maximum utility" for alternative cost-effectiveness 
combinations. 

5. Units for Effectiveness. 

3 

Karl Menger has shown that the variables of physics such as 
work, heat, energy, acelleration, etc., cannot be regarded as the 
class of numbers; nor can they be regarded as the class of physical 
entities. They must be regarded as the class of pairs (n, E) such 
that one element of the pair is a real ntmiber and the other is an ele- 
ment from the class of physical entities. 

Effectiveness can be considered as a physical entity. In fact, 
it is a characteristic of a physical system in this discussion. It 
cannot be expressed as only a number any more than it can be expressed 

3 

Menger, Karl, On Variables in Mathematics and Natural 
Sciences, British Journal for the Philosophy of Science, Vol. no. 18, 
1954, p. 135. 
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as only a physical entity, if the expression is to be useful in 
quantitative analysis • 

Three commonly used expressions of effectiveness may appear 
as contradictions to this idea. These are, (1) Probability, (2) 
Reliability, and (3) Ratios. All of these expressions imply the “unit” 
of complete or total effectiveness and they must include a lucid 
description of this ”unit“ to be meaningful. For a proposed measure 
of effectiveness to be placed on the ratio scale and expressed in 
units of U (say p.s.i), the mission of the system must be expressable 
in U and, in addition, the number of these units U necessary to 
achieve the mission must be known. 

Probability is commonly understood to represent the frequency 
of some specified event in relation to the number of opportunities 
for this event to occur. The statement that the probability of obtain- 
ing "heads" on any one toss of a coin is 1/2 means that the frequency 
with which this event will occur (over a long series of trials) is 
1/2 of the opportunities afforded. If probability is used as a measure 
of effectiveness we are saying that the system will exhibit complete 
effectiveness with the stated frequency. It is necessary to under- 
stand exactly how this "complete effectiveness" event is characterized 
before the probability statement can assume any meaning. For example, 
suppose "probability of kill" is used as the measure of effectiveness 
for some system. The implicit assumption is that each time the system 
exhibits the "kill event" it is completely effective for that trial, 
and further, if it exhibits the kill event each and every time over a 
long series of trials, it is completely effective (can accomplish its 
mission with certainty). The unit of effectiveness is the "kill event" 
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and the nvunber of these units necessary for complete effectiveness 
must equal the number of trials. 

Reliability is just that terminology applied to the probability 
of accomplishing a mission, and has the same interpretation as a 
probability statement. 

A ratio can be used in the same manner as probability, except 
that it is usually an "a posteriori" statement rather than an "a priori" 
statement. A ratio can also be interpreted as the magnitude of some 
system characteristic relative to some standard magnitude. Used as a 
measure of effectiveness, this implies that the "standard magnitude" 
of the characteristic will endow complete effectiveness. 

6. Measurability and Sub-Optimization. 

The usual task of quantitative analysis is the improvement of 
decisions at relatively low levels (efficiency "in the small"). 

Optimum decisions at low levels do not, however, imply an optimal 
solution to higher level structural decisions. If the Navy attempts 
to optimize the design of an aircraft carrier, taking the inclusion of 
a nuclear power plant as given, then it runs the risk of sub-optimiza- 
tion within a frame work in which the type of power plant is not 
assumed, but is a variable of the analysis. Suppose we have three 
sub-systems. A, B, and C, which are components of a larger system S, 

The effectiveness of system S is a function of the effectiveness of 
the sub-systems. Each of the sub-systems has a mission which, in 
combination , contribute to the mission of S. Optimizing the design 
of the sub-systems, according to some criterion that seems appropriate 
to their individual missions will not, in general, optimize the 
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criterion appropriate to the composite system S. 

Because of the sub-optimization problem, there has been a marked 
effort within the Defense Department to define systems in a broader, 
more inclusive manner. If the optimization process is applied correctly 
to the broader systems, sub -optimization among components is (in 
theory) avoided. It might seem that this problem could be entirely 
eliminated by defining the "system** as the complete defense establish- 
ment, and performing the optimization directly on this "system**. 

There are any number of obvious reasons why this would be an impossible 
task; but there is one, somewhat more subtle, difficulty that is not 
immediately obvious. This difficulty tends to dominate the process 
even at relatively low levels, and has to do with expressing the 
effectiveness element of the optimization criterion. 

6.1 Optimization. 

Any optimization process implies an operation on two or more 
characteristics of the system. If we optimize the mix of sand and 
cement to obtain the strongest concrete, the two elements are the **mix** 
and the strength of the concrete. A further requirement is that some 
functional relationship exist between these elements. Here, the 
strength of the concrete is a function of its **mix**. Alternatively, 
we can optimize the strength of the concrete subject to some cost 
constraint. If both the cost and strength of the concrete are functions 
of the mix, then this **mix** can be the parameter which relates cost 
to strength. In military systems analysis the two elements of the 
optimization process are generally the mix of elements (or design) 
and the effectiveness; or else the cost and effectiveness. It would 
be difficult to imagine an optimization function that did not include 
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the concept of effectiveness (althovigh the terminology might be 
different) , The concept of effectiveness contains the purpose of 
conceiving or building the system, and optimization Implies an opera- 
tion concerned with the purpose of the system. The first step In 
planning an optimization criterion Is the selection of the elements 
that the process Is to operate on. 

6.2 Measurability 

As the system becomes more broadly defined, envisioning the 
mission as a unique entity becomes more difficult. It may be necessary 
to state the mission In broad subjective terms, or even as a set of 
"sub", or alternative, missions. Examples of broadly defined missions 
are : 

(1) Provide a deterrent against enemy aggression, 

(2) Provide a retaliatory capability, 

(3) Provide a strategic nuclear delivery capability and 
an aerospace research vehicle. 

These missions suggest no easily conceived, unambiguous, entity which 
can be used as the second element of the measurement set. To obtain 
this element, and a measurement that can be Included In the optimiza- 
tion process, some characteristic of the system Is chosen that con - 
tributes to the achievement of Its mission. Unfortunately, the 
effectiveness of the system usually derives from additional character- 
istics In combination with the one measured. The difficulty of 
combining this set of characteristics Into a single element that can 
be logically Included In the optimization process Increases as the 
system becomes more broadly defined. 



20 



6.3, Stab -Optimization of Characteristics in Broadly Defined Systems 



When the effectiveness of a system is represented by a measure- 
ment of a contribut ing characteristic, rather than by a measurement 
on the set of all contributing characteristics, and this measurement 
is included in the optimization process, we are actually sub -optimizing 
between characteristics just as we sub-optimized between components in 
narrowly defined systems. 



7. System Model, 

A system can be modeled in the following way to indicate the 
manner in which effectiveness is derived from the basic resources 
employed in the system, A hypothetical missile system is used to 
provide representative examples for the notational elements. 



Let Xj^, x^, ****> , represent the basic resources used in 

the system. These can be single, 
initial, requirements or a flow per 
unit time . 



For example: 



Xj^ = tons of steel plate 

x^ = gallons of propellent 

x^ = mmiber of personnel 

x^ = megatons of explosives 

x^ = number of electronic components 

• • • 

• • • 

• • • 

X = etc, 

r 
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Let , i = 1,2,3, •••*, s, j = 1,2,3, 

i~Vi 

j production process operating on the 

Let (a, ,) be the matrix of the a. .*s 

i»j 

For example: 

a- - = cut the steel plate to dimension D. 

i» i 

a. ^ = form the steel plate into cylinder C. 

a. - = apply I’ed paint to the steel plate, 

• • • 

• • • 

• • • 

a- = etc. 
l,s 

Now if the Xj^»s, i ■ 1, 2, 3 ****,r are considered to constitute a 

— 4 

row vector X , and we form the combination ; 

X - Y 

then Y is a colimm vector having components: 
yp Y2* y3» ****»ys » 

where is a basic component (or unit) of the system. 

For example : 

y^^ = missile airframe 

y 2 = guidance system 



, r represent the 

,th 

i resource. 



The elements of X are related to those of (a. .) just as in 

1 > J 

ordinary vector multiplication, but these elements are not combined 

through ordinary multiplication. The element of (a ) "operates on" 

— 



its corresponding element in X to form an element of Y. The nature of 
this operation will be evident in context. 
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propulsion system 



y 

y 



3 

4 



warhead 



y = etc. 

Let u = 1,2,3, ♦•••,s, V = 1,2,3, ••••, t represent the 

til 

V tactic, employment, or environmental condition 

operating on the basic component. 

and (b ) be the matrix of the b ’s. 
u,v u,v 

For example : 

b- - = use two warheads on each airframe. 

1 j 1 

bl 2 ” protect each missile with a silo. 

b- ^ = representative characteristic of enemy antimissile 

^ defense. 



Now since Y is a column vector, form the combination; 

(b ) = C 

u,v 

Then C is a column vector having components; 

C C C •••• p 

1* V V ^ t 
th 

where is the h "basic” system characteristic. 
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For example 



probability that a missile reaches the antimissile 
defense area, given that it is launched, 

range of early warning radars 




c^ = reliability of guidance system 
c^ = warheads damage radius. 



c^ = etc. 

It is assumed that the c^’S are such that adequate, qxxantitative 
expression can be obtained for them. 



7.1 System Characteristics. 

In relating the h = 1,2,3, ****» t, to a single expression 

for effectiveness, it is helpful to utilize set notation. 

Let the set of all Cj^*,s, h = 1,2,3 ••**,t, be denoted by 
where the superscript refers to the "level" of the elements. Upon 
adding the superscript to the elements we have: 



= cjl>, c <», cll>, •••■, 



Now there will exist e sub-sets of where e-<t that have 

the property of completely describing the effectiveness of the 
system, if suitable functions are defined on these sets. 

Let j = 1,2,3,****, e , denote the such sub-set of 
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represent the set 



Let C 



(2) ^ (2) (2) (2) 

1 ’ 2 ’ 3 ’ 



.( 2 ) 



of"second level" characteristics whose elements are functions of the 



S<l)«s. 



This "second level" set of characteristics represents a manner 
of describing the system's effectiveness through a smaller, and more 
general, set of parameters. The lowest, or first, level of character- 
istics are all those numerous, and distinct attributes of a system 
contributing to its effectiveness. The elements of the "second level" 
set are combinations of these basic attributes that describe a slightly 
more general system characteristic. For example; suppose two basic 
(or first level) characteristics of a missile system are speed, and 
burn time. One element of the "second level" set would be the range 

of the missile as a function of these two basic characteristics. 

( 2 ) 

(i.e., ~ (speed) (burn time)) 

Then: 



= f. ), j = 1,2,3, , e , where 

th 

f. - is the function that relates the j set of first 

j,i 

th 

level characteristics to the j second level character- 
istic . 

Reviewing the notational scheme we have: 




the 



.th 
J , 



k-level, characteristic of a system. 



c(k) . 



the set of k-level characteristics completely describ- 
ing the system's effectiveness. 



s(k) . 

J 



the sub-set of 



f 






th 

the function relating the j sub-set of k-level 
characteristics to the jth (k + 1) -level characteristic. 
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Then; 

j = 1,2,3, •••*, e 

where e is the general "termination" of the index j. 

Now e is a "strictly decreasing" step function of k, i.e., as 
we express the effectiveness of a system in terms of higher level 
characteristics, the nuinber of these characteristics necessary to 
describe fully the effectiveness decreases . Finally, there will 
exist some k for which e “ 1. This single characteristic, then, 
fully describes the effectiveness of the system and, if suitable 
restrictions are imposed on the fj ^»s and c^^^'s, is a measurement 
of the effectiveness. These restrictions will be discussed later. 

In common with many notational schemes or models, the notation 
here is much more difficult than the idea that the model is intended 
to convey. The following diagrammatic representation of the mathe- 
matical model is presented to augment the notation. (Figure 1) 



7,2 Properties of the c 



(k), 

J 



s and f . ,8, 
J 



w,. 



The properties that must be possessed by the c^ '*e and 

f, .*B depend on the objective of the effectiveness measurement, 

J 

(see section 2.1) These properties are listed below in connection 
with analysis objectives. 



(1). If the measure of effectiveness is required only for the 

determination of equality; the Cj^^ , j = 1,2,3,****, t can be any 

measurement suitable for the nominal scale , and the set of f, , ’s are 

J 

required only to provide a one-to-one substitution. 
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(2) If the measure of effectiveness is required only for an 
"ordering" among alternatives; the Cj^^’s can be any measurement 
suitable for the ordinal scale , and the f^ j^'s can be any monotonic 
increasing functions. 

(3) If the measure of effectiveness is required for the 

determination of marginal rates of increase; the can any 

measurement suitable for the interval scale , and the fj ^^’s can be 
any linear functions whose constants need not be known. 

(4) If the measure of effectiveness is required as one of the 

elements in a cost-effectiveness combination that is to be compared 

for maximum utility; the c^^^’s must be "absolute" measures on a 

ratio scale , and the f. . *s must be linear with all constants known. 

J 

This linearity does not refer to the individual variables, but rather 
to the set of variables over which the function is defined. 



8, Measurability of the c^^^’s 

We assume that the basic characteristics of the system, which 

are denoted c^^\ are easily conceived, conceptual entities. Now 

there are two alternative methods of obtaining the next highest level 

set of characteristics: (1) direct physical measurement through some 

type of experimental testing, and (2) analytical modelling of the set 

of functions f. , , j • 1,2,3,****, e . Both of these tasks become 
j » * 

more difficult as the level (k) of the characteristics increases. 
Because of this Increase in difficulty, some level (k) is reached 
such that it is either impossible, or we are unwilling, to express 
the (k+1) -level of characteristics with any degree of certainty. 

Since the termination of the index J ( e ) is a function of k, there 
will exist e (k) k-level characteristics that contain all information 
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contributing to the effectiveness of the system* We are now presented 

with the problem of finding some function E, defined over the e(k) 

(k) 

elements of C , that will provide a single, quantitative expression 
for the effectiveness of the system. The difficulty of finding or 
formulating this function will increase as the number of variables 
(e(k)) increases. This ”trade-off" between measurability of individual 
system characteristics, and difficulty of obtaining the effectiveness 
function defined over this set (of characteristics) is encountered 
in every system analysis. The ability to resolve successfully this 
trade-off will also depend on the generality of the system definition 
and mission; and this in turn, affects the sub-optimization problem 
between systems. This interdependence of concepts is indicated in 
Figure II. 

From the preceding discussion (and Figure II), it can be seen 
that an important consideration in the system and mission definition 
phase should be a satisfactory **trade-off” between the sub -optimization 
among systems, and that among the characteristics of these systems. 



9. The Effectiveness Function. 



Let E represent the effectiveness of a system. 
Then: 



E = E(cW, cf), • 



**, c^^^), where: 
’ e ’ 



k is the highest level of measurable characteristics and 
e is the number of these characteristics necessary to 
completely describe the effectiveness. 
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Decrease Sub -optimization 
Among Separately Defined 
Systems 




FIGURE II 



30 



M, C* Heuston and G, Ogawa have listed 5 properties that this function 
must possess, and that will provide an ordering relationship comparable 
to the ordinal scale. 

(1) Its domain of definition is that part of euclidean e-space 

satisfying the condition that a point is in that set when all its 

(k) 

components (i.e., the c^ are all non-negative. This non-negative 

requirement can always be satisfied through a simple translation of 
the origin. " 

(2) Its range (i.e., the value of the function) is also non- 
negative. 

(3) When it is set equal to some constant, the resulting 
contour will define a hypersurface. 

(4) Its first partial derivatives are all positive. This means 
that as any one characteristic is increased (all others remaining 
constant) the effectiveness of the system will increase. 

(5) It is strictly quasi-concave- a property that can be 
shown to be equivalent to the **law of diminishing marginal returns”. 

This means that as any one characteristic is increased (all others 
remaining constant) the effectiveness will increase, but at a decreasing 
rate . 

These properties may provide some insight into the nature of this 
function, but are of little help in its construction. This task 
is usually (and necessarily) left to a decision maker who constructs 
the function in the form of his personal utility function. Assuming 

^Heuston, M, C, and Ogawa, G., Observations on the Theoretical 
Basis of Cost-Effectiveness, Journal of the Operations Research Society 
of America, Vol. 14, No. 2, p. 242. 
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that the decision-maker * s objectives are consistent with national 
objectives, the analyst's only concern is that the niimber and nature 
of characteristics presented to the decision maker are such that a 
well-formed, consistent utility function can be defined over this 
set (of characteristics). Whether E is considered to be some 
mathematical relationship or a utility function, the difficulty in 

(k) 

obtaining a “good” E is dependent on the number of elements in . 

9.1 A Mathematical Model of E . 

A common model that is found in a wide range of ”ef fectiveness 
literature” possesses all but number 5 (quasi-concave) of the proper- 
ties proposed above. This model provides a framework on which to 
apply relatively simple utility functions of only one variable. 



This model has the appeal of simplicity, but possesses serious short- 
comings when applied over a wide range of characteristic values. 

Its primary application is envisioned as ’’ordering” the effectiveness 
of several systems possessing identical types of characteristics. 
Unless this expression is envisioned as absolute, no suggestion is 
given concerning the comparison of. systems not possessing identical 
type characteristics. 











where the w^’s are arbitrarily assigned ’’weighting factors” 
that represent the utility of the individual c^^^’s. 



32 



10 . 



Conclusion. 



From this analysis of the concept of system effectiveness, the 
following list of tasks, considerations, and decisions should be 
followed in progressing from system definition to a quantitative 
criterion of its value. 

1. The mission should be defined as broadly as possible, 
consistent with some concept of how its ability to achieve this 
mission can be expressed quantitatively. 

2. The system designed or envisioned to accomplish this 
mission should be explicitly defined out to some "boundary". This 
boundary must separate the system from its environment, and contribu- 
tions to this mission from other elements or systems (outside this 
boundary) is incidental and outside of their own missions. 

3. A criterion for judging the value of the system, or for 
choosing between alternative systems or designs must be formulated, 
and/or. 



4. a method of optimizing the design or choice of the system 
must be devised. 

5. Based on the optimization method chosen, certain types of 
measurements must be obtained for a complete set of characteristics 
at the highest level possible. 

6. Depending on the number of elements in this set, a method 
of expressing the effectiveness of the system as a function of these 
elements must be designed, if this cannot be obtained with the 
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desired confidence of its correctness, 

7. the mission must be re-defined in such a manner that the 
effectiveness can be more confidently expressed. 
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APPENDIX I 



A Simplified, hypothetical missile system. 



Description: 100 ballistic missiles, silo protected, with associated 

early-warning radar system and command and control 
system. 

Mission: To deter the enemy from a premptive strike. 

Resources : 



■ steel plate 
^2 = propellent 

Xj “ explosives 
x^ » electronic components 
Xj = concrete 
Xg “ labor 

x^ » operating personnel 



Production proce 

®11 

®21 

^31 

%1 

^51 

^1 

^71 



sses : 

= form steel plate into cylinder C. 
= mix propellent into formula F 
= 0 

“ assemble guidance systems G 
• 0 

= assembly processes 
“ 0 
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Basic system element: 




/=/ 



= missile airframe 

(complete except for 
warhead) 



Production processes: 



12 



22 



form shape B 
0 



^32 ~ shape explosives into shape S 



42 



52 



62 



72 



= assemble fusing mechanism 



= 0 



assembly 

0 



Basic system element; 



1-7 

y (xia^z) 



= warhead assembly 






Production processes: 



^13 


■ 


form radar reflector R 


^23 




0 


®33 


= 


0 


®43 


= 


assemble radar transceiver 


®53 


= 


build radar sites 


^63 


= 


assembly and construction 


^73 


= 


staff with operating crew 
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Basic system element; 



^3 “ 


i=7 

^ ' IXldl 3^ “ radar early-warning system 

i'i 



Production processes: 



®14 ’ 


■ “24 ° “34 ° “54 ■ " 


^44 ’ 


= assemble communication equipment 


^64 ‘ 


= assembly and construction 


^74 ’ 


• staff command 6c control system with operating crew 



Basic system element; 



^4 “ 


/ (^iCUi) ® command and control sub- 

Z — . system 

4« / 



Production processes; 



®15 


= form silo hardware 


^25 ’ 


“ 0 


^35 


= 0 


®45 


= 0 


®55 


= build silos 


®65 


= assembly and construction 


^75 


= staff silo with launch crews 



Basic system element: 

L=? 



^5 “ 


" silo 
L--1 
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Then: 



(xj^,x2,x^,x^,x^,x^,x^) 



^11 ^12 ’^15 



21 



‘31 



71 



’^75 



r 



^2 

^3 

^4 



= airframe 

= warhead 

= early-warning 
system 

= command & control 



y^ = silos 




Tactics and environmental matrix. 

Enumerating the elements of (b^ is not enlightning because 
of the generality necessary. This matrix imposes tactics, procedures, 
and environmental conditions on the system elements producing the 
basic system characteristics. 



Then: B 



,( 1 ) 

'1 

,( 1 ) 

'2 

,(1) 

'3 

,(1) 

'4 

.( 1 ) 

'5 

.( 1 ) 

'6 

.a) 



.(1) 

'8 

.( 1 ) 



burn time of propulsion system. 

flight profile parameters. 

accuracy of guidance system. 

probability of penetrating anti- 
missile de fense . 

reliability of airframe. 

reliability of guidance system. 

range of early warning radar. 

reaction time of silo launch crew. 

reaction time of command and control 
system. 
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reliability of fusing mechanism. 



c 



( 1 ) 

10 



c 



( 1 ) 

11 



fusing height. 



c 



( 1 ) 

12 



overpressure distribution of warhead. 



c 



( 1 ) 

13 



blast resistance of silo. 



Second level set of characteristics. 
Sub-sets of the set are 



heigher level characteristics. 



.(1) 

^1 


- 




Then; 


c<2) . f 


(gd) \ 
1,1^^1 ^ 


o(l) 

^2 


= (c<^> 

iC4 . 


cd) e<^>) 

C5 » Cg ; 


Then; 


. f 


2,1^®2 ^ ^ 


c(l) 

^3 


= (c<^> 

ic? , 


cd) c<^>) 


Then; 


c<2) » f 

C3 r 


3, r ^3 •' 


o(l) 

^4 


- 


> 


Then; 






cd) 


. ( 

13 


) 


Then: 


(2) 

"5 ‘ 





now chosen to represent a set of 



* (burn time, flight profile, 
accuracy of guidance system) 

= C,E.F of missile at specified 
targets 



“ (probability of penetration, 
airframe reliability, guidance 
system reliability) 

® probability of missile reaching 
target . 

= (radar range, silo launch time, 
c&c launch t ime . ) 

® probability that missile can be 
launched if attacked. 

*= (fusing reliability, fusing 
height, overpressure) 

•* damage radius centered on 
explosion point. 

= (blast resistance of silo.) 
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damage radius necessary for 
attacking missile. 



All the elements in C 



can be obtained, either through experimental 



( 2 ) 



testing or by obtaining f. - analytically, 

J 5 i 



Third level set of characteristics. 



( 2 ) 

Sub -sets of C can now be chosen to represent the set, 

f 3) 

(C ) of third level characteristics. 

^ ” ( C.E.P. damage radius ) 



Then: ^ 



cf>, of) 



Then: ^ 



(expected damage at specified 
targets.) 

(probability of missile reaching 
target, probability that missile 
can be launched, damage radius 
necessary for attacking missile) 

probability that missile will 
reach target area. 



( 3 ) 

These elements of C cannot be obtained through experimental testing, 
but the functions 2 ’ ^2 2^ constructed with reasonable 

confidence . 



Effectiveness function. 



We now have a set of characteristics containing only two, well 
defined, elements which completely describes the effectiveness of the 
system. 



E 



E 





) 



If we believe, as has been assumed here, that the ability of this 
missile system to deter the enemy from conducting a premptive strike 
is a function of these two elements only, any information concerning 
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E will contribute to obtaining a measurement for the effectiveness. 

( 3 ) ( 3 ) 

Since c^ and c^ are both measurements on a ratio scale, it is 
possible to express the effectiveness of the system in absolute terms. 

hrb 

( 3 ) (3) 

If c^ ' < a and/or C 2 ^ ^ » where a and b are some minimvim 

valvie of the characteristics, then we can say that the effectiveness 

will be zero. We may be able to establish two other constants p, and 

(3) (3) 

q such that if; c^ p and q the effectiveness will be 

complete or 100%. These constants establish the end points of a 

(3) 

ratio scale and we require the effectiveness as a function of c^ 

( 3 ) 

and C 2 when: 

a < c^^^ < p and b •< c^^^ < q 

From this point we can assign the problem to a decision maker's 
utility function or apply some model such as eq. 2 . 
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